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SUMMARY 

In  previous  papers,!  <L  have  applied  the  functional  equation 
approach  of  dynamic  programming  to  the  study  of  variational 
problems  associated  with  the  Sturm-Llouvllle  equation  of  second 
order  with  real  coefficients.  In  this  way,  we  were  able  to 
obtain  the  dependence  of  the  Green's  function  upon  the  Interval 
length.  Prom  this  ,we  obtained  the  corresponding  dependence  of 
the  characteristic  values  and  the  charact.rlstlc  functions,  and 
similar  results  for  vector-matrix  systems. 

To  apply  the  same  general  techniques  to  the  study  of 

equations  with  complex  coefficient-*  . 

^  0061  * icients ,  we-ttsB  min-max  variation. 

It  Is  shown  that  this  method  can  be  applied  rigorously. 
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FUNCTIONAL  EQUATIONS  IN  THE  THEORY  OF 
DYNAMIC  PROGRAMMING- -XI I:  COMPLEX 
OPERATORS  AND  MIN- MAX  VARIATION 

Richard  Bellman 
R.  Sherman  Lehman 

1 .  Introduction 

r  1 

In  111  ,  we  applied  the  functional  equation  approach  of 
dynamic  programming  \2  j  to  the  study  of  the  variational  problem 
associated  with  the  Sturm-Llouville  equation 

(1)  fp(x)u')'  +  (r(x)  +  Ag(x))u  -  0,  u(a)  -  u(l)  -  0, 

where  p,  q,  and  r  are  real.  In  this  way,  we  were  able  to 
obtain  the  dependence  of  the  Oreen'e  function  upon  the  end¬ 
point  a.  From  this  result,  we  obtsiined  the  dependence  of  the 
characteristic  values  and  characteristic  functions  upon  a. 
Corresponding  results,  the  Hadaward  variational  formulas,  were 
presented  for  partial  differential  equations  in  3j  • 

In  order  to  apply  the  functional  equation  technique,  we 
were  required  to  assume  that  p,  q,  and  r  were  real.  In 

r  1 

4J ,  we  indicated  briefly  how  mln-max  techniques  could  be  used 
to  study  the  corresponding  problems  for  complex  functions.  In 
this  paper,  we  wish  to  present  the  full  argument  for  a  parti¬ 
cular  class  of  equations  of  the  foregoing  form. 

2.  Boundary  Value  Problem 

Let  us  consider  the  following  boundary  value  problem: 

U"(x)  +  q(x)u(x)  =  v(x),  (a  <  x  <  b), 

U(a)  -  0,  U(b )  -  0, 


(1) 
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where  q(x)  and  v(x)  are  complex-valued  functions  continuous 
on  the  closed  Interval  ,a,b],  If  the  Sturm- Llouville  problem 
obtained  by  replacing  q  by  ^q  does  not  have  7\  =•  1  as  a 
characteristic  value,  then  the  boundary  value  problem  has  a 
unique  solution  given  by  the  equation 

(2)  U(x)  -/b  K(x,y,a)v(y)dy 

a 

where  K(x,y,a)  Is  the  Oreen's  function  of  the  problem.  We 
shall  regard  b  as  fixed  and  study  the  dependence  of  the 
Oreen's  function  upon  a. 

In  order  to  use  an  approach  suggested  by  dynamic  program¬ 
ming,  we  shall  find  It  convenient  to  study  the  system 

(3)  u"(x)  +  q(x)u(x)  -  v(x ) ,  (a  <  x  <  b), 
u(a)  «  c,  u(b)  -  0. 

A  solution  of  this  system  can,  under  the  above  assumptions,  be 
expressed  In  the  form 

(4)  u(x )  -  U(x)  +  c^(x) 

where 

(5)  ^"(x)  +  q(x)^(x)  -  0,  (a  <  x  <  b), 

^(a)  »  1,  ^(b)  *  0. 

Since  the  problem  (3)  Is  associated  with  a  complex  linear 
differential  operator  which  Is  symmetric,  we  can  obtain  a 
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corresponding  variational  problem.  Let  u(x)  -  u^(x)  ■+  iUg(x), 
q(x)  -  q1(x)  +  iq2(x),  v(x)  -  v^x)  +  iv2(x),  with  u1#u2#q1# 
q2#vl,v2  r«al-valued  functions,  and  let  c  -  c,  +  ic2  with 
c1  and  c2  real  numbers.  The  system  (3)  is  then  equivalent 
to  the  real  system 

(6)  ^"(x)  +  q^xju^x)  -  q2(x)u2(x)  -  v^x), 
u2"(x)  +  qx(x)u2(x)  +  qgUJu^x)  -  vg(x), 
ux(a)  -  cx,  u2(a)  -  c2,  u^b)  -  u2(b)  -  0. 

We  consider  the  Integral 

J<VU2>  -/>b  {-  ul'2  +  u2 ' 2  +  «1U12  -  2Viu2  -  ^“e2 

Ql 

-  2u1v1  +  2u2v2j(ix 

and  consider  the  variational  problem  of  determidhg 

(7)  f(a,c,,c  )  -  max  min  J(u.,u?), 

U1  u2 

where  the  maximum  is  taken  over  piecewise  continuously  differen¬ 
tiable  functions  for  which  u^a)  «  o^,  u^b)  -  0,  and 

the  minimum  is  taken  over  piecewise  continuously  differentiable 
functions  ug  for  which  u2(a)  =  c2,  u2(b)  -  0. 

The  equations  (6)  are  the  Euler  equations  for  the  varia¬ 
tional  problem  (7)  and  the  functions  and  u2  yielding  the 

max  min,  if  they  exist,  provide  a  solution  of  the  system  (6). 

To  see  this,  we  let  T^Cx)  and  be  arbitrary  piecewise 


pw- 
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continuously  differentiable  functions  for  which  ^(a)  -  ??2(a) 
-  ?!«»>  *■  ??2(b)  «  0  and  let  and  be  positive  real 

numbers .  Then 


o 

J(ul  +  ei?^i'u2  +  £2^2^  "  J^uii,u2^  +  2eiAl  +  2e2A2  +  61  A11 


+  2e1e2Ai2  +  e2  a22. 


where 


A1  mc/)b  W  +  <iiui^l  “  q2U2?l  "  vl?l)dx» 
a 

A2  “</;b  ^2^2  ”  q2U1^2  “  qlU2^2  + 

a 

Ail  -c/711  (-  V2  +  ^i2)41' 

a 

*12  ”  ~(d  q2?1^2<lx' 


*22  V7”  (92'2  "  l^s2)4*' 


Integrating  by  parts  we  obtain 


Ax  •(/^b  /7?1(x)[u1"(x)  +  q1(x)u1(x)  -  q2(x)u2(x)  -  v1(x)Jdx, 

& 

A2  "  “V^  ^2^X^[U2"^X^  +  q1(x)ug(x)  +  qgUJu^x)  -  v2(x)Jdx. 

a 

Consequently  (u^Ug)  is  a  stationary  point  for  J  if  and 
only  if  u^  and  ug  yield  a  solution  of  the  boundary  value 
problem  (6). 


i 
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In  order  to  assure  the  existence  of  the  max  min  appealing 
In  (7),  It  Is  sufficient  to  assume  that  q^(x)  <  0  for 
a  <  x  <  b.  It  then  follows  that  If  '^(x)  ft  0  and  ??2(x)  ?  °# 
then  Ai;l  <  0  and  A 22  >  0.  Consequently,  the  max  min  In  (7) 
will  exist  and  also 


(8) 


f(a,c1,c2)  -  min  max  J(u1,uJ2) 


u2  U1 


where  the  minimum  and  maximum  are  taken  over  the  same  classes 
of  functions  as  for  (7). 

We  remark  that  If  q^(x)  is  not  negative  on  the  endre 
interval  [a,b  then  one  can  employ  analytic  continuation  as 


in 


to  reduoe  the  problem  to  one  for  which  the  variational 


argument  is  applicable. 


3.  Interchanging  min  and  max  Operators 

Let  X1#  X2,  Y1#  and  Yg  be  sets;  and  let  x1#  y1# 

and  y2  be  variables  ranging  over  these  sets.  Let  ^(x1,x2,y1,y2) 
be  a  real-valued  function  defined  for  all  values  of  the 
variables.  Assuming  that  all  of  the  maxima  and  minima  mentioned 
exist,  we  shall  prove  the  following  lemma. 

LEMMA  1.  If 


(1) 

*  max 

max 

min 

min  < 

[(x1,x2,y1,y2) 

X1 

X2 

yl 

y2  ' 

-  min 

min 

max 

-1 

max  j 

[(x1,x2#y1,y2) 

*1 

y2 

X1 

X2 

then 
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(2)  max  min  max  min  px1#x2,y1,yg)  * 

X1  yl  x2  y2 

and 

(3)  *ln  max  min  max  i(x,  ^x^y,  ,y0)  -  p. 

yl  X1  y2  x2  '* 

Proof.  We  use  repeatedly  the  faot  that 

(4)  max  min  ^(x,y)  £  min  max  ^(x,y) 

x  y  y  x 

for  any  function  V  of  variables  x  and  y  ranging  over  sets 
X  and  Y,  provided  the  maxima  and  minima  exist.  (See  5  ,p.lO 

We  have 

(3)  max  max  min  min  i  <£  max  min  max  mini 

*1  x2  yx  y 2  xi  yi  x2  y2 

£  min  max  max  min  j> 
yl  X1  x2  y2 

<;  min  max  min  maxi 

y^  x1  y2  xg 

£  min  min  max  max  <i. 

y  1  y2  xi  x2 

By  assonptlon  the  t*o  expressions  at  the  ends  of  this  string 
of  Inequalities  are  equal.  Hence  we  can  replace 
throughout  by  and  the  lemma  follows. 

We  remark  that  the  proof  shows  that  the  lemma  holds  even 
If  the  set  Xg  depends  upon  the  variable  x^  and  the  set  Yg 
depends  upon  the  variable  y^. 
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Lemma  1  can  be  regarded  as  a  statement  of  "the  principle 
of  optimality"  for  r.  particular  type  of  two-stage  game.  In 
this  interpretation  is  the  value  of  the  two  person  game. 

The  first  player  chooses  the  strategy  x^  for  the  first  stage 
and  the  strategy  x2  for  the  second  stage.  The  other  player 
chooses  y^  for  the  first  stage  and  yg  for  the  second.  Por 
a  further  discussion  of  the  principle  of  optimality  for  multi¬ 
stage  games  we  refer  to  [2;p.29~]. 

4.  Application  of  Dynamic  Programming 

Using  an  approach  suggested  by  the  theory  of  dynamic 
programming,  we  shall  discuss  a  variational  problem  of  which 
the  problem  of  ^2  is  a  special  case.  Let  P  be  a  continuous 
function  of  five  variables.  Let 

(1)  f(a,c,fcpj  »  max  min  /?b  P(x(t),x,(t  ),y(t  ),y,(t  ),t  )dt 

yl  ^a 

-  min  max  ^  ?(x(t)#x1(t),y(t),y1(t),t  t 
7l  a 

where 

(2)  x(t)  -  c1  +(/)t  x1(s)ds, 

a 

y(t)  -  c p  +(/?t  yA»)<la, 

a 

and  the  maxima  are  taken  over  functions  x1(t)  which  are  piece- 
wise  continuous  on  the  interval  [a,b]  and  the  sdnima  are 
taken  over  functions  y^(t)  which  are  piecewise  continuous  on 
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the  Interval  j^a,bj.  The  assumption  that  the  max  min  Is  equal 
to  the  min  max  will  be  Important  In  the  following  discussion. 

In  addition  we  shall  assume  that  the  function  f  Is  continuously 
differentiable  and  that  the  functions  and  yielding 

the  min  max  are  continuous. 

Let  A  be  a  small  positive  number.  We  let  max^  denote 
the  maximum  taken  over  piecewise  continuous  functions  x^ 
restricted  to  the  interval  a, a  +  A  ,  max^j  denote  the 

maximum  over  piecewise  continuous  functions  x^  restricted  to 
the  Interval  [a  +  A,b  ,  and  similarly  min^  denote  the 
minimum  over  piecewise  continuous  y^  on  'a, a  +  A  ,  and 
minn  the  minimum  over  piecewise  continuous  y.^  on  ja  +  A,bj. 
We  have  by  Lemma  1 

(3)  f(a,o,,c0)  -  max  max  min  min  V/7b  Pdt 

1  d  I  II  I  II  a 

=»  max  min  max  min  i  Fdt  +  Fdt  ( 

I  I  II  II  lca  ( a+A 


«  max  min  i  ,/?a+A  Pdt  +  max  mln^/^  Fdt 
I  I  Ha  II  II  a+A 


-  max  min  i,/^a+A  Fdt  +  f(a  +  A,x(a  +  A),y(a 
I  I  <ua 

This  states  algebraically  the  principle  of  optimality  for 
multistage  games — at  each  stage  am  optimal  policy  Is  one  leading 
to  am  optimal  continuation. 

Using  the  assumed  continuity  of  P  amd  continuous 
differentiability  of  f,  we  obtain  for  A  — 0 
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(4)  f(a#o1#c2)  -  max  min  £AF(e1,x1(a),c2,x2(a) ,a)  +  f(a,c1,c2 

+  4  -g(».c1,c2)  +  ••*1(a)f£-(..o1,o2) 

+  Ayi(a>|^-(a,c1,c2)  +  o( A )] , 

and  henae  for  A  — >  0 

(5)  0  -  max  min  £F(e]L,x1(&),e2,x2(a),&)  +  §£  +  x i(a)|^ 

+  yi<a)ff2  +  0(1  >]• 

Hera  It  Is  tempting  to  Ignore  the  o(l)  term  and  aay  that  in 
the  limit  a  choice  of  x^  and  y.^  over  the  Interval  a, a  +  A 
amounts  to  a  choice  of  x1(a)  and  y^(a).  Let  ua  Justify  this 
argument.  Let  0(t)  be  an  arbitrary  continuous  function  on 
the  Interval  a,b]  such  that  0(a)  ■  0.  Since  we  are  assuming 
the  existence  of  a  continuous  solution,  let  us  take  the  maximum 
and  minimum  not  over  all  piecewise  continuous  functions,  but 
over  all  continuous  functions  of  the  form 

(6)  xx(t)  -  x1(a)  +  xx(t),  |x1(t)|  ^  0(t), 

yx(t)  -  yx(a)  +  yx(t),  |y1(t)|  £  0(t). 

Clearly,  by  choosing  0  appropriately  we  can  have  the  solution 
of  our  original  problem  In  the  admissible  class. 

Abbreviating  by  setting  c^'  =  x^a),  c2'  -  y^a)  and 

interchanging  minimum  and  maximum  operators  according  to 
Lemma  1,  we  have  for  A  — *  0 
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(7) 


max 
o 


21 

3a. 


=  max 
o 


+  °i’  l^  +  V  §-2  +  °<1>; 

*  ^U.V'Cz'V' a)  +  Is +  V  Hr  +  °2'  H: 

C.'  1  2 


+  max  min  o(l)? 
A  "  '  A  #t)  5 


xx(t)  yx(t) 


Because  of  the  assumed  properties  of  the  solution,  we  can  re¬ 
strict  the  point  (c1,,c2l)  to  a  bounded  region.  Then,  If  we 
Investigate  the  o(l)  term  more  closely,  we  see  that  the 
restrictions  (6)  guarantee  that  the  term  o(l)  is  uniformly 
small  for  all  functions  x^  and  y1  for  which  |  ( t ) |  <r  p(t), 

|$2(t)|  £  3(t).  Consequently,  we  obtain 

(8)  0  -  max  min  |P(c1,oJL  •  ,c2,c2*  ,a)  +  ^(a»c1»c2) 

C1 ’  C2* 


+  c 


1 


||-(a,o1,o2) 


V 


9f_ 

3c, 


(a,cx 


,C2 


Specializing  this  result  to  the  variational  problem  of  ^2, 

we  have 


(9) 


0 


.  2  p  p 

max  min  j-  c1'  +  c2'  +  c1  q-^a)  -  2c.jC2q2(a) 

cl'  c2 ' 

p 

-  c 2  q1(a)  -  2c1v1(a)  +  2c2v2(a) 

+  la^a,cl,c2^  +  cl'  f£^a'cl,c2) 

+  C2 '  lc^^a,Cl,C2^  * 


It  is  easily  seen  that  the  max  min  of  this  quadratic  function 
is  attained  for 
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(10) 


1  3f  „  ,  1  ?f 

°2 


Substituting  these  values  Into  (9)  we  obtain  the  partial 
differential  equation 

(n)  "  la^a,0l'c2^  "  i^^a,Cl#C2^1 2 3  “  i^^^a'°l'c2^2 


-  2c1v1(a)  +  2c2v2(a) 

+  (o^  -  c2  )q1(a)  -  2c1c2q2(a). 


5.  Alternate  Approaoh 

Another  way  of  formulating  the  variational  problem  of  ^2 
Is  gotten  by  observing  that 

(1)  jfu^Ug)  -  R»(/)b  [-  (u'(x))2  +  q(x)(u(x))2 

fit 

-  2u(x)v(x)j  dx. 

Similarly  the  partial  differential  equation  (4.11)  oan  be 
written 

(2)  -  |5^a,cl,c2)  "  Micj)  +  2cv(a)  +  c2q(a) 

where  c  -  c^  +  ic2. 

The  following  lemma  will  be  used  in  the  next  section. 

LEMMA  2.  If  P(x,y)  is  continuous  for  a  <  x  <  b,  a<y< 
if  F(x,y)  =  F(y,x),  and  if 

(3)  Re(y^bi/?b  F(x,y)v(x)v(y)dxdy  -  0 

a  a 
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for  every  contlnuoua  function  v,  then  F(x,y)  vanlaheu 
Identically. 

If  F(x)  It  contlnuoua  for  a  <  x  <  b,  and  If 
Re/?b  P(x)v(x)dr  -  0 

for  every  continuous  function  v,  then  P(x)  vanlahea 
Identically . 

Proof.  In  caae  P  la  a  real- valued  function  and  (3) 
holds  for  every  real-valued  contlnuoua  v,  the  first  assertion 
of  the  lemma  waa  proved  In  [lj.  Let  F(x,y)  »  F1(x,y)  +  lF2(x,y), 
v(x)  -  +  lv2(x)  with  F,,  ?2,  v1#  v2  real-valued 

functions.  We  have 

(4)  0  -  Re  /}bc/)b  P(xfy)v(x)v(y)dxdy 

a  a 

n/b/b  jP1(x*y)  [v1(x)v1(y)  -  v2(x)v2(y)] 
a  a  L 

-  P2(x,y)  [v1(x)v2(y)  +  v2(x)v1(y)  j^dxdy. 

Taking  v2(x)  ■  0,  we  obtain 

^/)by?b  pi(x,y )Vl(x)Vl(y)dxdy  -  0 
a  a 


for  all  contlnuoua  real-valued  functions 


lemma  of 
obtain 


s  0*  Next,  taking 


v^.  Hence  by  the 
v2(x)  =  vx(x),  we 


-  2c/?b</?b  P2(Xfjr)vl(x)vl(y)dxd 7  "  °' 
a  a 
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and  thus  Fg(x,y)  -  0.  Thi&  completes  the  proof  of  the  flret 
assertion  of  the  lemma .  The  second  assertion  can  be  obtained 
as  a  corollary  by  taking  F(x,y)  -  lF(x)F(y). 

6.  Variation  of  the  Qreen's  Function 

Now  let  us  use  the  partial  differential  equation  (3*2)  to 
study  the  variation  of  the  Oreen's  function  K(x,y,a)  with  a. 
Let  u(x)  -  u1(x)  +  iu2(x)  be  the  function  which  yields  the 
max  min  In  (2.7)  for  given  a  and  o.  Writing  u  -  U  + 
and  Integrating  by  parts,  we  find 

(1)  -  /)h  u ' 2dx  -y1*  jU(U"  +  2c^")  +  c2#"{dx  +  cV'(a). 

A  A 


Hence,  using  the  differential  equations  U"  +  qU  -  v  and 
+  q^  ■  0,  we  obtain 

(2)  f(a,c1,c2)  -  **</’'*  (*“  u'2  +  qu2  ”  2uv)dx 

& 

-  Re  |  -6/}b  U(x)v(x)dx  -  2c ^(x)v(x)dx 

fit  & 

+c2^'(a)J, 

and  hen<;e  by  (2.2) 


(3)  f(a,olfc2)  -  Re  j  K(x,y,a)v(x)v(y)dxdy 

-  2c  ^(x)v(x)dx  +  c2^'(a)?. 


Using  the  differential  equation  (5.2),  we  obtain  upon 
equating  terms  independent  of  c1  and  c2. 
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R*/Vb|i(x'y^)v(x)v(y)  a  (H*/V(x)v(x)dx): 


a  a 


-  (ia^/7*  ^(x)v(x)dx)' 


Re  /)b  /)b  )v(x)v(y )dxdy 

u&  a 


for  every  continuous  function  v.  Because  of  the  symmetry  of 
the  Oreen's  function,  we  can  apply  Lemma  2  to  obtain 

f”(x,y,a)  «  ^(x)^(y). 

Also 

Pb  jrfvdx  -  /)b  ^(u"  +  qU)dx  »  pb  u(^"  +  q^)dx  -  U'(a)^(a) 
°a  6a 

--  U'(a), 


and  hence 

(4)  (/)b  ^(y)v(y)dy  - |^(a,y,a)v(y)dy, 

&  fit 

for  every  continuous  function  v.  Hence ,  by  Lemma  2, 

(5)  *(y)  »-f|(a,y,a). 

Thus 

PK /  \  2K/  _  \9&t  ~  \ 

^(x,y,a)  -  ^(a,y,a)^(x,a,a). 


(6) 
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to  obtain  an  equation  for  the  resolvent  as  a  function  of  a. 
Specializing  the  values  of  7)  aa  in  1  ,  we  obtain  in  this 
way  variational  formulas  for  the  characteristic  values  and 
characteristic  functions. 

Finally,  let  us  mention  that  as  in  jl  it  is  easy  to 
obtain  analogous  results  for  the  vector-matrix  version  of  (l). 


p- 2083 
16 


REFERENCES 


1.  R.  Bellman  and  R.  S.  Lehman,  "Functional  equations  In  the 

theory  of  dynamic  programming — X:  resolvents, 
characteristic  functions  and  values,"  Duke  Math.  J., 
vol.  27,  I960,  pp.  55-70. 

2.  R.  Bellman,  Dynamic  Programming,  Princeton  Univ.  Press, 

Princeton,"  n."7.",  “1957  • - 

3.  R.  Bellman  and  H.  Osborn,  "Dynamic  programming  and  the 

variation  of  Oreen’s  functions,"  J.  Math,  and  Meoh., 
vol.  7,  1958,  pp.  81-86.  "  . . 

4.  R.  Bellman  and  R.  S.  Lehman,  "Functional  equations  In  the 

theory  of  dynamic  programming — IX:  variational  analysis, 
analytlo  continuation,  and  Imbedding  of  operators," 

Proc.  Nat.  Acad.  Scl.  USA,  vol.  44,  1958,  pp.  905-907. 

5-  J.  C.  C.  Me  Kinsey,  Introduction  to  the  Theory  of  Qames, 
McGraw-Hill  Book  (Jo.,  Inc.,  Wew  York,  1952. 


